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ABSTRACT 



The Special Supplemental Nutrition Program for Women, 
Infants, and Children (WIC) is a federally funded program providing 
supplemental food and nutrition services to lower- income pregnant, 
breastfeeding, and postpartum women and also serves infants and children up 
to age 5 who are at nutritional risk. Included in these services are 
nutrition education, breastfeeding promotion and support, and referrals to 
health and social services. As directed by the William F. Goodling Child 
Nutrition Reauthorization Act of 1998, this report: (1) identifies the number 

and nature of recent studies that have examined the effectiveness of 
nutrition education, breastfeeding promotion and support, and referral 
services; and (2) summarizes research information on the effectiveness of 
these services. Information was obtained through relevant databases and 
consultation with WIC program staff and other program stakeholders. Twelve 
demonstration and seven impact studies conducted after 1995 were examined. 
Most studies focused on breastfeeding promotion and support. Despite some 
methodological limitations, the findings of the demonstration studies suggest 
that certain types of interventions, such as providing breastfeeding support 
in the hospital following delivery, have the potential to be more effective 
than the usual WIC interventions. The more effective strategies may cost more 
than usual WIC approaches. The findings of the seven impact studies provided 
few insights into the effectiveness of specific services. Their findings were 
severely limited by methodological constraints, including the use of outdated 
and poor-quality data. This report concludes with a recommendation that the 
Department of Agriculture improve demonstration research by incorporating 
relevant cost information. Six appendices include information on the 
methodology, bibliographies of the demonstration and impact studies, and a 
table delineating major findings, scope, and limitations of the demonstration 
studies. (Contains 23 endnotes.) (KB) 
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Accountability * Integrity * Reliability 



United States General Accounting Office 
Washington, D.C. 20548 



March 30, 2001 

The Honorable Richard G. Lugar 
Chairman 

The Honorable Tom Harkin 
Ranking Member 

Committee on Agriculture, Nutrition, and Forestry 
United States Senate 

The Honorable John A. Boehner 
Chairman 

The Honorable George Miller 
Ranking Democratic Member 
Committee on Education and the Workforce 
House of Representatives 

The Special Supplemental Nutrition Program for Women, Infants and 
Children (WIC) is a federally funded nutrition assistance program 
administered by the U.S. Department of Agriculture’s (USDA) Food and 
Nutrition Service (FNS). This program provides supplemental food and 
nutrition services to lower-income pregnant, breastfeeding, and 
postpartum women. The program also serves infants and children up to age 
5 who are at nutritional risk. The WIC program serves a monthly average of 
7.3 million individuals, including about 47 percent of all infants bom in the 
United States. In fiscal year 2000, the Congress appropriated $3.9 billion to 
fund WIC, $2.8 bilhon of which was used to provide food benefits and $1.1 
billion of which was used for nutrition services and program 
administration. The nutrition services part of the program includes 
nutrition education, breastfeeding promotion and support, and referrals to 
health and social services. 

A substantial body of research has examined the overall effectiveness of 
the WIC program, but little is known about what research shows regarding 
the effectiveness of specific WIC nutrition services. This report (1) 
identifies the number and nature of recent studies that have examined the 
effectiveness of three WIC services — nutrition education, breastfeeding 
promotion and support, and referral services — and (2) summarizes what 
the research shows about the effectiveness of these specific nutrition 
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Results in Brief 



services. This report is the fourth in a series providing information on 
various aspects of WIC nutrition services as directed by the William F. 
Goodling Child Nutrition Reauthorization Act of 1998 (RL. 105-336).* 

To identify recent studies that examine the effectiveness of each WIC 
nutrition service, we searched relevant databases, such as the National 
Technical Information Service, Sociological Abstracts, and Wilson Social 
Science Abstracts. We also consulted with USDA WIC program staff and 
other program stakeholders including officials from the National 
Association of WIC Directors. Initially, we identified more than 200 
published WIC studies dealing with various aspects of the WIC program. In 
order to focus on recent research, we eliminated from consideration 
research published prior to 1995. We also eliminated any study that did not 
specifically address the effects of at least one of the three WIC nutrition 
services, whether nutrition education, breastfeeding promotion and 
support, or referral services. We included research that falls in two general 
categories: demonstration studies and impact studies. The demonstration 
studies typically attempt to assess whether special interventions, such as 
hospital visits to promote breastfeeding, more effectively achieve program 
objectives than the usual WIC services. In contrast, impact studies attempt 
to determine whether a WIC nutrition service, such as breastfeeding 
promotion and support, improves WIC participants’ outcomes compared to 
similar individuals who do not participate in the WIC program. 

We conducted our work between May 2000 and March 2001 in accordance 
with generally accepted government auditing standards. Appendix I 
provides a more detailed description of the methodology we used to 
conduct this work. 



We identified 19 studies published since 1995 that examine the 
effectiveness of WIC nutrition services. Twelve of the 19 are demonstration 
studies; seven are impact studies. Most of the studies — seven of the 
demonstration studies and four of the impact studies — focused on 
breastfeeding promotion and support. 



^ Food Assistance: Financial Information on WIC Nutrition Services and Administrative 
Costs (GAO/RCED-00-66, Mar. 6, 2000); Food Assistance: Activities and Use of Nonprogram 
Resources at Six WIC Agencies {GKO/RC'EDfX)-2ii2, Sept. 29, 2000); Food Assistance: 
Performance Measures for Assessing Three WIC Services {GkOIQ\-‘i2Q, Feb. 28,2001). 
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Despite some methodological limitations, the results of the 12 
demonstration studies suggest that certain types of nutrition service 
interventions, such as providing breastfeeding support services in the 
hospital after delivery, have the potential to be more effective than theusual 
WIC interventions. Our analysis of the demonstration studies suggests that 
the more effective strategies may cost more than usual WIC approaches. 
However, only one of the studies specifically addressed the potential costs 
of new interventions. The results of the seven impact studies provide few, if 
any, insights into the effectiveness of specific WIC nutrition services. The 
results of the impact studies are severely limited by methodological 
constraints, including the use of outdated and poor-quality data. 

We are making a recommendation to USDA aimed at improving 
demonstration research by incorporating relevant cost information. We 
provided a draft of this report to USDA for its review and comment. In 
commenting on the draft report, USDA generally agreed with the report 
and its recommendation. The agency provided some technical comments, 
which we incorporated as appropriate. 


Background 


WIC, which began as a 2-year pilot program in 1972 and was authorized as a 
permanent program in 1974, is part of the nutrition safety net available to 
low-income women and their children. FNS provides annual cash grants for 
food benefits and nutrition services to fund program operations at 88 state- 
level WIC agencies (including agencies in all 50 states, the District of 
Columbia, American Samoa, the Commonwealth of Puerto Rico, Guam, the 
U.S. Virgin Islands, and 33 Indian Tribal Organizations). Some of these 
state-level agencies — those that operate the program at both the state and 
local levels — retain all of their federal WIC grants. Most state-level 
agencies, however, retain a portion of their grants and pass the remaining 
funds to over 1,800 local WIC agencies. 

In fiscal year 2000, about $2.8 bilhon in federal program funds were used to 
provide food benefits to participants. Typically food benefits are in the 
form of vouchers or checks that participants can use to obtain approved 
foods at authorized retail food stores. An additional $1.1 billion in federal 
funds were used for nutrition services and program administration. 
Program administration includes, among other things, activities related to 
accounting and record keeping, outreach, monitoring and financial audits, 
and general management. Nutrition services include activities related to 
determining participants’ eligibility and issuing food benefits, as well as the 
following: 
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• Nutrition education: WIC offers classes, counseling, and other activities 
to teach participants about proper nutrition, positive food habits, and 
the prevention of nutrition-related problems. 

• Breastfeeding promotion and support: To promote breastfeeding, WIC 
offers individual and group counsehng sessions at WIC clinics or the 
hospital. Breastfeeding support can include telephone or in-person 
consultation with breastfeeding mothers. 

• Referral to health care and social services: WIC agencies provide 
participants with information on health care and social services and 
refer them to providers including immunization clinics and the Food 
Stamp and Medicaid programs. 

By law, spending for nutrition education and breastfeeding promotion and 
support activities combined must equal at least one-sixth of a state’s total 
annual expenditures for nutrition services and administration plus a target 
amount for breastfeeding promotion and support that is established by FNS 
at the beginning of each fiscal year.^ There is no minimum spending 
requirement for referral activities. 

Over the past 20 years, government agencies such as USDA,® the Centers 
for Disease Control and Prevention, and GAO,"* * as well as universities and 
private research organizations, have conducted a substantial body of 
research on the effects of the entire WIC program. Some of the 
accumulated body of WIC research and evaluations provides nationwide 
assessments of WIC’s effects. Most of it has focused on the effect of 
program participation on birth outcomes and the nutritional status of 
program participants. USDA has a review under way describing and 
assessing research on the diet and health outcomes of its nutrition 
programs, including WIC. The results of this review, set for release later 



^In fisccil year 2000, the state agencies’ targets for breastfeeding promotion and support 
were determined by multiplying $23.92 by the average number of pregnant and 
breastfeeding participants during the last 3 months for which there were fineil data (May- 
June 1999). 

®In 1998, the Agriculture, Rural Development, Food and Drug Administration, and Related 
Agencies Appropriations Act of 1998 (P.L. 105-86) effectively transferred primary 
responsibility for research funding on food assistance and nutrition programs — including 
WIC — from FNS to USDA’s Economic Research Service. Previously, USDA had spent about 
$3.5 million annually on such research. 

* Breastfeeding: WIC’s Efforts to Promote Breastfeeding Have Increased (GAO/HRD-94-13, 
Dec. 6, \QQW)\Early Intervention: Federal Investments Like WIC Can Produce (GAO(\iYCD-Q2- 
18, Apr. 7, 1992) 
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this year, will provide detailed information on over 70 studies, most of 
which examine WIC’s effects on birth outcomes or on the nutrition status 
of participants. While the USDA review will not focus on the impacts of 
specific nutrition services, it will include studies that examined the WIC 
program’s effects on the initiation and duration of breastfeeding and the 
immunization status of children. These two health-related outcomes are 
directly linked to two of the three nutrition services addressed in this 
report — breastfeeding promotion and support and referral services. 
However, because the USDA review is generally focused on overall 
program impacts, its report probably will not include descriptions or 
assessments of many of the demonstration studies included in this report. 



Most of the Recent 
Research Evaluates 
Demonstrations of 
Special Interventions 



The 19 studies we identified included almost twice as many demonstration 
studies as impact studies. Of the 12 demonstration studies, 3 look at special 
interventions in nutrition education, 6 look at special interventions in 
breastfeeding promotion and support, and 3 look at special interventions in 
referral to health and social services. Of the seven impact studies, one 
examines nutrition education, four assess breastfeeding promotion and 
support services, and two evaluate WIC health referrals.® 



Most of the studies have a relatively limited geographic scope. Among the 
12 demonstration studies, 11 are at the substate level. They generally study 
multiple WIC sites and/or multiple counties, but without sufficient 
sampling rigor to draw valid statewide conclusions. The results of one 
demonstration study are generalized to an entire state. Among the seven 
impact studies, three are at the substate level, one is statewide, and three 
are national in scope. 



The 19 studies received funding from various sources. Table 1 provides 
details on project funding for the 12 demonstration studies and 7 impact 
studies. 



®One impact study — Fox and others — is primarily a nutrition education evaluation, so -we 
count it as such. However, it also contains information on WIC breastfeeding programs and 
their effects on initiation and duration. 
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Table 1: Funding for Demonstration and Impact Studies 


Source of funding 


Number of 
demonstration 
studies funded^ 


Number of impact 
studies funded^ 


USDA 




1 


2 


U.S. Department of Health and Human 
Services 




1 


0 


Centers for Disease Control and 
Prevention 




5 


0 


State departments of health 




4 


1 


Universities and research institutes 




1 


2 


Private organizations 




1 


1 


Unspecified 




1 


3 



“These columns do not sum to 12 and 7, respectively, because some studies received funding from 
multiple sources. 



Appendix II provides detailed information on the funding sources for the 19 
studies reviewed for this report. 



Special Interventions 
Improve Participant 
Outcomes but 
Research Says Little 
About the 
Effectiveness of 
Individual Nutrition 
Services 



While all 19 studies suffer from methodological limitations, those 
limitations have varying consequences. Despite their limitations, the results 
of the 12 demonstration studies suggest that special interventions have 
some potential to improve nutrition service effectiveness over WIC 
interventions typically used; though, based on our analysis of these studies, 
it appears that additional resources may have to be committed to achieve 
this added effectiveness. However, the methodological limitations of the 
seven impact studies enable them to provide only very limited information 
on the effects of any one nutrition service. Appendixes III and IV contain 
lists of the demonstration and impact studies, respectively, reviewed for 
this report. 



Demonstration Studies 
Indicate Some Special 
Interventions Improve 
Participant Outcomes 



The 12 demonstration studies evaluate a range of different special 
interventions. To varying degrees, all were more effective than usual WIC 
interventions. Examples of the special interventions include the following: 

• Breastfeeding promotion and support. Gross and others evaluated 
special interventions designed to encourage breastfeeding among 
African-American WIC participants.® Mothers in the three special 
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intervention groups were provided a motivational video or peer 
counseling or a combination of the video and counseling. Mothers in the 
control group received the standard WIC service that incorporated 
encouragement and support to breastfeed and brochures about 
breastfeeding during discussions about infant feeding. Mothers in the 
special intervention groups were twice as likely as mothers receiving 
the standard WIC infant feeding education to be breastfeeding 8 weeks 
and 16 weeks after giving birth, even accounting for factors that could 
increase breastfeeding duration, such as prior breastfeeding experience. 

• Health referrals. Birkhead and others evaluated two special 
interventions designed to increase the number of WlC-eligible children 
who receive measles immunizations.^ The special interventions included 
having WIC staff escort children to an on-site immunization clinic and a 
food voucher incentive in which WIC staff provided only a 1-month 
supply of vouchers to parents, rather than the usual 2-month supply, 
until the parents provided documentation that their children’s 
immunizations were up-to-date. The standard WIC immunization 
referral consisted of notifying parents that immunizations were due, 
providing information on the benefits of immunizations, and providing 
the names and telephone numbers of local health facilities that 
immunize children. Children at escort sites were about five times more 
likely to be immunized than children at standard referral sites; children 
at voucher incentive sites were about three times more likely to receive 
immunizations. 

• Nutrition education. Havas and others evaluated a special intervention 
designed to increase WIC participants’ consumption of fruits and 
vegetables.® The special intervention — Maryland’s “5-A-Day” program — 
was a series of three 45-minute group sessions taught by former WIC 
participants, or “peer educators,” that incorporated special visual 
materials and included direct mailings to participants. The standard 



®Susan M. Gross, Laura E. Caulfield, Margaret E. Bentley, and others, “Counseling and 
Motivational Videotapes Increase Duration of Breastfeeding in African-American WIC 
Peuticipants Who Initiate Breastfeeding,” Journal of the American Dietetic Association, Vol. 
98, No. 2 (1998), p. 143. 

’Guthrie S. Birkhead, Charles W. LeBaron, Patricia Parsons, and others, “The Immunization 
of Children Enrolled in the Special Supplemental Food Program for Women, Infants, and 
Children (WIC),” Journal of the American Medical Association, Vol. 274, No. 4 (1995), p. 312. 

*Steven Havas, Jean Anliker, Dorothy Damron, and others, “Final Results of the Maryland 
WIC 5-A-Day Promotion Program,” American Journal of Public Health, Vol. 88, No. 8 (1998), 

p. 1161. 
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service generally included less than 10 minutes of nutrition education 
conversation between WIC staff and participants when they picked up 
their food voucher every other month. Compared to participants 
receiving the standard WIC nutrition education program, participants 
exposed to the special intervention displayed a significant increase in 
nutrition knowledge and in the consumption of fruits and vegetables. 

Each of the demonstration studies we reviewed suffers from 
methodological limitations that, while not invalidating the study’s findings, 
should be taken into account. The limitations we identified are common in 
studies that attempt to assess the extent to which social or health program 
interventions — not other factors — are responsible for changes in program 
participants’ behaviors or health. The major methodological limitations of 
the demonstration studies we reviewed include the following: 

• Lack of control group. To help isolate the effects of an intervention, an 
evaluation study must compare people receiving the special 
intervention to similar people receiving standard WIC services. The 
difference between these groups can provide insight into whether the 
special intervention is more effective than standard WIC practice. Not 
having such a comparison obscures the relationship between the 
intervention and participant outcomes. Four of the studies we reviewed 
had a weak research design associated with a lack of control group. For 
example, Hoekstra and others attempted to evaluate the effectiveness of 
a new voucher incentive program in increasing WIC children’s rates of 
immunization.® However, the researchers did not compare the group 
receiving the special intervention to a group receiving standard services. 
Instead, they compared three special intervention groups to themselves 
at different points in time over a period of 14 months. Without a control 
group that does not participate in the voucher intervention, it is difficult 
to attribute any changes the researchers noticed to the special 
intervention. 

• Inappropriate data analysis techniques. The analytic techniques used in 
a study must suit the available data and the research design — in 
particular, they should be selected for their ability to help isolate the 
effects of the intervention. Three of the demonstration studies we 



“Edward J. Hoekstra, Charles W. LeBaron, Yannis Megaloeconomou, and others, “Impact of 
a Large-Scale Immunization Initiative in the Special Supplemental Nutrition Program for 
Women, Infants, and Children (WIC), ” Joum^ of the American Medical Association, Vol. 
280, No. 13(1998), p. 1143. 
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reviewed used questionable analytic techniques. For example, Havas 
and others found that a special peer counselor program was effective in 
increasing nutrition knowledge and the consumption of fruits and 
vegetables. To reach this finding, Havas and others compared the 
average fruit and vegetable consumption of the special “5-a-Day” group 
to the average fruit and vegetable consumption of a group exposed to 
the standard WIC nutrition education. The comparison, which examined 
the linkage between demographic characteristics, such as race, and fruit 
and vegetable consumption, did not take into account the simultaneous 
influence of other characteristics, such as education level, on 
consumption. Without an analysis technique, such as multiple 
regression, that can account for the influence of several factors at once, 
■ determining the extent to which the observed differences in fruit and 
vegetable consumption are the result of the “5-A-Day” program is greatly 
complicated. 

• Selection bias. Ideally, study participants should be randomly assigned 
to intervention and control groups to ensure that all participant 
characteristics will, on average, be the same from one group to another. 
A selection bias exists if the two groups differ in some systematic way. 
Selection bias makes it more difficult to attribute an observed difference 
in outcomes between the two groups to any one factor, such as the 
intervention. Six of the demonstration studies we reviewed have a 
possible selection bias. For example, Tuttle and Dewey examined the 
influence of a new, culturally sensitive breastfeeding education 
intervention on the initiation and duration of breastfeeding among 
Hmong WIC participants in Northern California. " However, the 
study’s research design depended on women to volunteer to participate 
in the study— the women self-selected themselves to participate in the 
special intervention. Thus, those choosing to participate in the special 
intervention may have shared characteristics (for example, an already 
existing inclination to breastfeed) that did not exist in those women 
who chose not to participate. If present, selection bias could lead the 
researcher to overstate the benefits of the special intervention. 

• Low response rate/missing data. To substantially reduce data reliability 
concerns, experts agree that under most circumstances researchers 
need at least 75 percent of the people they ask to participate in a study 



‘“Cynthia Reeves Tuttle and Kathryn G. Dewey, “Impact of a Breastfeeding Promotion 
Program for Hmong Women at Selected WIC Sites in Northern California,” Journal of 
Nutrition Education, Vol. 27, No. 2 (1995), p. 69. 

"Most Hmong living in the United States today came from Laos. 
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to agree. Excessive missing data, or poor response rates, may skew 
research findings. Missing data or poor response rate was a limitation in 
five of the demonstration studies. For example, Ahluwalia and others 
evaluated five new breastfeeding interventions, and attributed 
significant improvements in breastfeeding initiation to them.*^ However, 
the database employed by the study contained breastfeeding initiation 
data for only 52 percent of the women in the sample. 

• Measurement error. For an analysis to produce reliable results, the 
measures in the analysis must be accurate. Measurement error is the 
difference between a measured value and its true value. Five of the 
studies have potential measurement errors. For example, Shaw and 
Kaczorowski sought to examine the effectiveness of a peer coimseling 
program on breastfeeding initiation and duration by asking new mothers 
to recall interactions with breastfeeding peer counselors that took place 
at the time of birth. Mothers were interviewed 6 weeks to 6 months 
after giving birth. If memory lapses occurred, new mothers may have 
incorrectly recalled their dealings with peer counselors, thereby 
potentially introducing measurement error into the data. 

Appendix V shows the major findings and methodological limitations of 
each of the 12 demonstration studies. 

Our analysis suggests that the effective interventions described in the 
demonstration studies may cost more than standard WIC approaches. For 
example, most breastfeeding special interventions were specifically 
designed to increase the amount of counseling and support provided to 
prenatal and postpcutum women. Although only one of the demonstration 
studies provided information about the additional costs associated with 
such interventions, it is reasonable to expect that such one-on-one support 
will cost more than the standard WIC program. 

Two of the demonstration studies help to illustrate the linkage between 
resource commitment and results achieved. The first, Ahluwalia and 
others, which found that a hospital-based strategy providing bedside 



‘^Indu B. Ahluwalia, Irene Tessaro, Laurence M. Grummer-Strawn, and others, “Georgia’s 
Breastfeeding Promotion Program for Low-Income Women,” Pediatrics, Vol. 105, No. 6 
(2000), p. 85. 

'^Ehzabeth Shaw and Janusz Kaczorowski, “The Effect of a Peer Counsehng Program on 
Breastfeeding Initiation and Longevity in a Low-Income Rural Population,” Journal of 
Human Lactation, Vol. 15, No. 1 (1999), p. 19. 




Page 12 



13 



GAO-01-442 Research on WIC Nutrition Services 



support and counseling to women who had just given birth, was the most 
effective at increasing breastfeeding initiation rates out of five new 
strategies evaluated. This strategy clearly required more resources than the 
standard practice of providing counseling and brochures to participants 
during a visit to the WIC chnic. The second study, Weimer, which was 
funded by USDA, focused on a special intervention that was similarly 
resource-intensive. It reported that providing one-on-one support in the 
hospital after delivery, followed by an in-home visit within 72 hours of 
birth, increased breastfeeding duration. However, neither study provided 
any information about the additional costs needed to implement these 
special interventions. 

Only one of the 12 demonstration studies we reviewed, Hutchins and 
others, provided any information about the costs associated with the 
implementation of the special intervention. This study reported that 
vaccinations increased at sites providing vaccination screening and 
voucher incentives (until their children are immunized, a family must visit 
the chnic monthly — rather than every 3 months — to pick up WIC 
vouchers). This study uses what its authors term “crude” cost-effectiveness 
ratios to estimate the average cost for each additional child with up-to-date 
immunizations. These costs range between $30 and $73, depending on the 
number of enrolled children and rates of active participation. 



Impact Research Provides The seven impact research studies we reviewed provide few conclusive 
Very Limited Information on insights into the recent effectiveness of WIC breastfeeding promotion and 
the Effectiveness of WIC’s support, referral services, or nutrition education services. 

Individual Nutrition 
Services 



‘“^Sonja S. Hutchins, Jorge Rosenthal, Pamela Eason, and others, “Effectiveness and Cost- 
Effectiveness of linking the Special Supplemental Program for Women, Infants, and 
Children (WIC) and Immunization Activities,” Journal of Public Health Policy; VoL 20, No. 4 
(1999), p. 408. 
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Breastfeeding Promotion 
and Support 



Three of the four impact studies that focused on breastfeeding promotion 
and support — Schwartz and others/® Balcazar and others/® and Timbo and 
others‘^ — use old data from the 1988 National Maternal and Infant Health 
Survey. Although, according to FNS officials, this survey represents the 
most recent data available, much has changed in the program since 1988, 
including the characteristics of WIC participants and the emphasis the 
program places on breastfeeding. As a result, these studies’ findings shed 
little light on the program’s current effects. Although the fourth study, 

Wiemann and others, uses data from the mid-1990s, its limited scope, in 
terms of geography and participants, constrains the applicability of its 
findings. This study, with data collected from 684 adolescent mothers who 
gave birth at a hospital in Galveston, Texas, could have some specialized 
usefulness, but would have to be replicated at many other sites to provide 
insights into the broader effectiveness of WIC’s breastfeeding promotion 
and support services. In addition, since adolescent mothers comprise only 
about 11 percent of all WIC mothers, the study’s focus on them further 
compromises its more general usefulness. 

Taken as whole, the inconsistency in the findings of these four studies 
further limit their usefulness in assessing the effects of WIC’s breastfeeding 
promotion and support program. For example, Wiemann and others and 
Balcazar and others find that WIC enrollment is a significant factor in some 
mothers’ decision to bottle-feed, while Timbo and others, and Schwartz and 



‘^J. Brad Schwartz, Barry M. Popkin, Janet Tognetti, aind others, “Does WIC Participation 
Improve Breastfeeding Practices,” American Journal of Public Health, Vol. 85, No. 5 (1995), 
p. 729. 

'^Hector Balcazar, Catherine M. Trier, and Jose A. Cobas, “What Predicts Breastfeeding 
Intention in Mexican-American and Non-Hispanic White Women? Evidence from a National 
Survey,” Birth, Vol. 22, No. 2 (1995), p. 74. 

‘‘Babgaleh Timbo, Sean Altekruse, Marcia Headrick, and others, “Breastfeeding Among 
Black Mothers: Evidence Supporting the Need for Prenatal Intervention,” Journal of the 
Society of Pediatric Nurses, Vol. 1, No. 1 (1996), p. 35. 

'^Constance M. Wiemann, Jacqueline C. DuBois, and Abbey B. Berenson, “Racial/Ethnic 
Differences in the Decision to Breastfeed Among Adolescent Mothers,” Pediatrics, Vol. 101, 
No. 6 (1998), p. 11. 

‘°Susan Barlett, Melanie Brown-lyons, Douglas Moore, and others, WIC Participant and 
Program Characteristics 1998 {Nlexandna, Va.: USD A, Food and Nutrition Service, 2000), p. 
38. 
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Referral Services 



others, conclude that WIC participation increases breastfeeding. No 
consistent message emerges from the studies. 

The two referral service impact studies have methodological constraints 
that, to varying degrees, limit their usefulness in assessing the effectiveness 
of WIC referral services. The first study, Suarez and others, using survey 
data from 30 counties in Texas, found that children who are enrolled in 
WIC are significantly more likely than children who are not enrolled to be 
up-to-date on their immunizations, regardless of other intervening factors 
such as the child’s age, ethnicity, or the family’s income.^® Although this 
study likely contains some measurement error, it provides at least limited 
evidence that WIC referral services are effective in increasing 
immunization rates.^^ In contrast, due to m^or methodological problems, 
the second referral study, McCunniff and others,^^ provides little useful 
information on the effectiveness of WIC referral services. This study is 
based on self-administered questionnaire data collected from a sample of 
mothers at three WIC sites in Kansas City, Missouri. The study found that 
when taking into account factors such as WIC referral, child’s age, 
household size, and availability of dental insurance, only the age of the 
child had a significant, independent effect on the likelihood that children 
will visit a dentist. There are two principal limitations to this study. First, 
almost 40 percent of the sampled children were younger than 1 year old. 
Because many children less than 1 year of age do not yet have teeth, they 
are much less likely to have made a dental visit, thus reducing the study’s 
ability to identify factors associated with dental visits other than age. The 
second mqjor limitation is the measurement error associated with the 
reliance on self-reported questionnaire data about visits to the dentist. The 
study did not attempt to verify questionnaire responses through a review of 
dental records. The authors suggested that such reviews would have 
increased the accuracy of the self-reported data. As a result of these 
serious methodological problems, it is likely that McCunniff and others has 



^"Lucina Suarez, Diane M. Simpson, and David R. Smith, “The Impact of Public Assistance 
Factors on the Immunization Levels of Children Younger Than 2 Years,” American Journal of 
Public Health, Vol. 87, No. 5 (1997), p. 845. 

^‘Measurement error is likely in Suarez and others because the immunization information 
provided by 28 percent of those interviewed was based on parents’ recollections of their 
children’s immunizations, rather than medical records or other documentation. 

“Michael D. McCunniff, Peter C. Damiano, Michael J. Kanellis, and others, “The Impact of 
WIC Dental Screenings and Referrals on Utilization of Dental Services Among Low-Income 
Children,” Pediatric Dentistry, Vol. 20, No. 3 (1998), p. 181. 



o 



Page 15 



16 



GAO-Ol-442 Research on WIC Nutrition Services 



Nutrition Education 



Conclusion 



only limited relevance to understanding the effectiveness of WIC referral 
services. 

The one study that primarily focused on the impact of nutrition education, 
Fox and others, also examined breastfeeding programs and their 
effectiveness.^® However, Fox and others was limited geographically, and 
had other limitations that reduce its usefulness in assessing the 
effectiveness of WIC’s nutrition education. For example, its scope was 
limited in that it focused on pregnant and postpartum women at six WIC 
sites, in three states. Within this limited context, the study describes 
program and participant characteristics; the nutrition services offered 
(including breastfeeding promotion and support); participants’ receipt of 
and satisfaction with these services; and changes in participants’ 
knowledge and behaviors between the time of prenatal WIC certification 
and 4 to 6 months postpartum. The study also attempted to assess the 
impact of WIC nutrition education on participants’ knowledge and 
behavior. Although the study concluded that participants’ nutrition 
knowledge and behavior improved significantly over the course of the 
study, attributing these changes to WIC is problematic because the study 
did not use a control group. Instead, Fox and others compared intervention 
groups at different points in time. The study also concluded that (1) WIC 
participation did not significantly increase breastfeeding initiation or 
duration; and (2) women’s decisions regarding infant feeding are strongly 
associated with intentions formed during pregnancy. 



Demonstration studies, despite some methodological limitations, provide 
program managers and policymakers with some useful information about 
the types of WIC nutrition service interventions that can have positive 
impacts on participants. However, only one recent demonstration study 
provides any information on the costs associated with implementing 
various interventions. Given the limited resources available to provide WIC 
nutrition services, information about the costs to provide effective services 
could play a critical role in managers’ decisions to implement the 
intervention and policymakers’ decisions about funding the intervention. 



^^Mary Kay Fox, Nancy Burstein, Jenny Golay, and others, WIC Nutrition Education 
Assessment Study: Final Report{PA^yiax\An&, Va.: U.S. Department of Agriculture, Food and 
Nutrition Service, 1999), p. vii. 
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Recommendation 



Agency Comments and 
Our Response 



In order to maximize the value of nutrition education, breastfeeding 
promotion £md support, and referral service demonstration and evaluation 
research funded by USD A, we recommend that the Secretary of Agriculture 
direct officials responsible for implementing such research to require that 
this research include an assessment of the costs associated with the special 
intervention being evaluated. 



We provided a draft of this report to the Department of Agriculture’s Food 
and Nutrition Service for review and comment. We met with Food and 
Nutrition Service officials, including the Acting Associate Deputy 
Administrator for Special Nutrition Programs. The agency officials 
generally agreed with the report’s findings and recommendation. However, 
the officials questioned why our recommendation did not address actions 
that WIC researchers should take to deal with some of the methodological 
limitations we identified in research evaluating the effectiveness of WIC 
services. We believe that USDA has a responsibility to ensure that the WIC 
and other nutrition program research it funds are of high quality. However, 
our review was not designed to examine USD As policies and procedures to 
ensure the quality of the research it funds or the practices it employs to 
promote high-quality research in studies it does not fund. As a result, we 
are not making any specific recommendations concerning how USDA 
might improve the quality of WIC research at this time. 

The officials also provided some technical changes and clarifications to the 
report, which we incorporated as appropriate. 



We are sending copies of this report to the appropriate congressional 
committees; interested Members of the Congress; the Honorable Ann M. 
Veneman, Secretary of Agriculture; the Honorable Mitchell E. Daniels, Jr., 
Director of the Office of Management and Budget; and other interested 
parties. We will also make copies available upon request. 
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If you or your staff have any questions about this report, please contact me 
or Thomas E. Slomba at (202) 512-7215. Key contributors to this report are 
listed in appendix VI. 




Robert E. Robertson 

Director, Employment, Workforce, and 
Income Security Issues 
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Appendix I 



Scope and Methodology 



To identify recent studies that examine the effectiveness of the Special 
Supplemental Nutrition Program for Women, Infants and Children (WIC) 
nutrition education, breastfeeding promotion and support, and referral 
services, we searched relevant databases, such as National Technical 
Information Service, Sociological Abstracts, and Wilson Social Science 
Abstracts. We also consulted with the U.S. Department of Agriculture 
(USD A) WIC program staff and other program stakeholders, including 
officials from the National Association of WIC Directors. Through this 
process, we identified 209 items published from 1988 through 2000 dealing 
with various aspects of the WIC program. To be used in our review, 
individual items had to meet each of the following criteria: 

• publication in a refereed medium (for example, a journal article, book or 
book chapter, USDA-issued report); 

• publication date of 1995 or later; 

• examination of one or more of WIC’s nutrition services (breastfeeding 
promotion and support, nutrition education, or health referrals); and 

• original analysis of a specific nutrition service’s effectiveness. 

Altogether, only 19 items met all four criteria. Many — 86 of the 190 items 
we rejected — were published prior to 1995, and therefore do not satisfy our 
definition of recent studies. (We established 1995 as the cutoff to enable us 
to better examine the program as it currently operates.) We eliminated the 
remaining 104 items because they did not meet one or more of our criteria. 
For example, some items appeared in our literature search as professional 
papers delivered at conferences; thus, they did not undergo any formal 
referee process. Other items were published as reviews or summaries of 
original research, but did not include any original research of their own. 
Some items do not focus on the effectiveness of specific WIC nutrition 
services. For example, one study examines the general effects of food 
programs — including WIC and other food assistance programs such as food 
stamps — on diet, but does not evaluate the effectiveness of specific WIC 
nutrition service programs. 

Once we narrowed the scope of our study, we met with staff in the USDA’s 
Food and Nutrition Service Office of Analysis, Nutrition and Evaluation to 
ensure that our methodology did not exclude any important studies. 
According to these officials, our approach successfully identified all of the 
major recent evaluation studies on WIC nutrition services. 

We then conducted detailed reviews of the 19 studies. These reviews 
entailed an evaluation of each study’s research methodology, including its 
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Scope and Methodology 



data quality, research design, and analytic techniques, as well as a summary 
of its major findings and conclusions. We also assessed the extent to which 
each study’s data and methods support its findings and conclusions. 
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Appendix II 



Research Funding Sources 







Study 


Funding source(s) 


Demonstration studies 


Ahluwalia and others, 2000 


Centers for Disease Control and Prevention 


Hutchins and others, 1 999 


Centers for Disease Control and Prevention 


Shaw and Kaczorowski, 1999 


Unspecified 


Abusabha and others, 1998 


New Mexico Department of Health 


Gross and others, 1998 


U.S. Department of Health and Human Services 


Havas and others, 1998 


National Cancer Institute; Maryland Department of 
Health and Mental Hygiene 


Hoekstra and others, 1998 


Centers for Disease Control and Prevention 


Weimer, 1998 


USDA/ERS 


Grummer-Strawn and others, 1997 


Centers for Disease Control and Prevention 


Reifsnider and Eckhart, 1997 


Beta Delta Chapter at Large; Sigma Theta Tau 


Birkhead and others, 1995 


Centers for Disease Control and Prevention 


Tuttle and Dewey, 1995 


California State Department of Health 


Impact studies 


Fox and others, 1999 


USDA/FNS 


McCunniff and others, 1998 


Unspecified 


Wiemann and others, 1 998 


National Institute on Drug Abuse; 
Hogg Foundation for Mental Health 


Suarez and others, 1997 


Texas Department of Health; Public Policy 
Research Institute, Texas A&M University 


Timbo and others, 1996 


Unspecified 


Balcazar and others, 1 995 


Unspecified 


Schwartz and others, 1 995 


USDA/FNS; Research Triangle Institute, University 
of North Carolina 
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Bibliography of Demonstration Studies 



Breastfeeding ' " 
Promotion and Support 



Ahluwalia, Indu B.; Irene Tessaro, Laurence M. Grummer-Strawn, and 
others. “Georgia’s Breastfeeding Promotion Program for Low-Income 
Women.” Pediatrics, Vol. 105, No. 6 (2000), pp. 85-91. 



Shaw, Ehzabeth, and Janusz Kaczorowski. “The Effect of a Peer Counseling 
Program on Breastfeeding Initiation and Longevity in a Low-Income Rural 
Population.” Journal of Human Lactation, Vol. 15, No. 1 (1999), pp. 19-25. 

Gross, Susan M., Laura E. Caulfield, Margaret E. Bentley, and others. 
“Counseling and Motivational Videotapes Increase Duration of Breast- 
Feeding in African-American WIC Participants Who Initiate Breast- 
Feeding.” Journal of the American Dietetic Association, Vol. 98, No. 2 
(1998), pp. 143-148. 

Weimer, Jon P. Breastfeeding Promotion Research: The ESAVIC Nutrition 
Education Initiative and Economic Considerations. Washington, D.C.: U.S. 
Department of Agriculture, Economic Research Service, 1998. 

Grummer-Strawn, Laurence M., Susan P. Rice, Kathy Dugas, and others. “An 
Evaluation of Breastfeeding Promotion Through Peer Counseling in 
Mississippi WIC Clinics.” Maternal and Child Health Journal, Vol. 1, No. 1 
(1997), pp. 35-42. 



Reifsnider, Ehzabeth, and Donna Eckhart. “Prenatal Breastfeeding 
Education: Its Effect on Breastfeeding Among WIC Participants.” Journal 
of Human Lactation, Vol. 13, No. 2 (1997), pp. 121-125. 

Tuttle, Cynthia Reeves, and Kathryn G. Dewey. “Impact of a Breastfeeding 
Promotion Program for Hmong Women at Selected WIC Sites in Northern 
California.” Journal of Nutrition Education, Vol. 27, No. 2 (1995), pp. 69-74. 



Nutrition Education Abusabha, Rayane, Cheryl Achterberg, and Jeannie McKenzie. “Evaluation 

of Nutrition Education in WIC.” Journal of Family and Consumer Sciences, 
Winter (1998), pp. 98-104. 

Havas, Stephen, Jean Anliker, Dorothy Damron, and others. “Final Results 
of the Maryland WIC 5-A-Day Promotion Program.” American Journal of 
PubUc Health, Vol. 88, No. 8 (1998), pp. 1161-1167. 
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ReferrS-lS Hutchins, Sonja S., Jorge Rosenthal, Paimela Eason, and others. 

“Effectiveness and Cost-Effectiveness of Linking the Special Supplemental 
Program for Women, Infants, and Children (WIC) and Immunization 
Activities.” Journal of Public Health Policy, Vol. 20, No. 4 (1999), pp. 408- 
426. 

Hoekstra, Edward J., Charles W. LeBaron, Yannis Megaloeconomou, and 
others. “Impact of a Large-Scale Immunization Initiative in the Special 
Supplemental Nutrition Program for Women, Infants, and Children (WIC).” 
Journal of the American Medical Association, Vol. 280, No. 13 (1998), pp. 
1143-1147. 

Birkhead, Guthrie S., Charles W. LeBaron, Patricia Parsons, and others. 
“The Immunization of Children Enrolled in the Special Supplemental Food 
Progrcim for Women, Infants, and Children (WIC).” Journal of the American 
Medical Association, Vol. 274, No. 4 (1995), pp. 312-316. 
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Bibliography of Impact Studies 



Breastfeeding 
Promotion and 



Fox, Mary Kay, Nancy Burstein, Jenny Golay, and others. WIC Nutrition 
Q , Education Assessment Study: Final Report. Neyi 2 iX\AndL,Na.. \'[].^. 

oUpport Department of Agriculture, Food and Nutrition Service, 1999. 



Wiemann, Constance M., Jacqueline C. DuBois, and Abbey B. Berenson. 
“Racial/Ethnic Differences in the Decision to Breastfeed Among 
Adolescent Mothers.” Pediatrics, Vol. 101, No. 6 (1998), pp. 11-23. 



Timbo, Babgaleh, Sean Altekruse, Marcia Headrick, and others. 
“Breastfeeding Among Black Mothers; Evidence Supporting the Need for 
Prenatal Intervention.” Journal of the Society of Pediatric Nurses, Vol. 1, 
No. 1 (1996), pp. 35-46. 

Balcazar, Hector, Catherine M. Trier, and Jose A. Cobas. “What Predicts 
Breastfeeding Intention in Mexican-American and Non-Hispanic White 
Women? Evidence From a National Survey.” Birth, Vol. 22, No. 2 (1995), pp. 
74-80. 

Schwartz, J. Brad, Barry M. Popkin, Janet Tognetti, and others. “Does WIC 
Participation Improve Breast-Feeding Practices?” American Journal of 
Public Health, Vol. 85, No. 5 (1995), pp. 729-731. 



R6f6rrEllS Nancy Burstein, Jenny Golay, and others. WIC Nutrition 

Education Assessment Study: Final Report. Alexandria, Va.: U.S. 
Department of Agriculture, Food and Nutrition Service, 1999. 

McCunniff, Michael D., Peter C. Damiano, Michael J. Kanellis, and others. 
“The Impact of WIC Dental Screenings and Referrals on Utilization of 
Dental Services Among Low-Income Children.” Pediatric Dentistry, Vol. 20, 
No. 3 (1998), pp. 181-187. 

Suarez, Lucina, Diane M. Simpson, and David R. Smith. “The Impact of 
Public Assistance Factors on the Immunization Levels of Children Younger 
Than 2 Years.” American Journal of Public Health, Vol. 87, No. 5 (1997), pp. 
845-848. 
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Demonstration Studies: Major Findings, 
Scope, and Major Limitations 





■ * 


study 


Major finding(s) 


Scope (timeframe) 


Major limitation(s) 


Nutrition education 


Abusabha and others, 
1998 


Nutrition education lectures and facilitated group 
discussions were more effective than brochures at 
increasing participants’ nutrition knowledge. 
Facilitated group discussion was more effective 
than brochures at increasing participants’ 
confidence in performing specific nutrition related 
behaviors. 


Seven WIC clinics in 
New Mexico (not 
specified) 


Missing data; 
selection bias 


Havas and others, 1998 


Consumption of fruits and vegetables increased 
after an education program consisting of a series of 
three 45-minute group sessions taught by paid peer 
educators and incorporating special visual 
materials and a direct mailing to participants. 


Sixteen WIC clinics in 
Maryland (not 
specified) 


Missing data; measurement 
error; inappropriate data 
analysis techniques 


Breastfeeding 


Gross and others, 1998 


The duration of breastfeeding among African- 
American WIC participants increased with peer 
counselor support or viewing promotional' 
breastfeeding videos. 


Four WIC sites in 
Baltimore, Md. (1992- 
1994) 


Selection bias; 
measurement error 


Ahluwalla and others, 
2000 


WIC participants increased the initiation of 
breastfeeding when exposed to (1) an enhanced 
education program with access to a hotline, (2) a 
free breast-pump loan program, (3) a hospital- 
based program with bedside support and 
counseling after delivery, (4) community coalitions, 
and, (5) peer counseling provided by former 
participants. 


State of Georgia ' 
(1992-1996) 


Missing data 


Weimer, 1998 


1 . Breastfeeding initiation and duration rates 
increased after volunteer peer counseling. 

2. Breastfeeding initiation and duration rates 
‘increased after paid peer counseling. 

3. Duration of breastfeeding increased after 
postdelivery contact with mother in hospital 
was followed up with support (including home 
visits) by a specially trained paraprofessional. 

4. Breastfeeding initiation and duration rates 
increased with culturally appropriate 
breastfeeding education provided in high 
school or WIC clinics. 


1. Two communities 
in Iowa (not 
specified) 

2. Six counties in 
Michigan (not 
specified) 

3. Five counties in 
North Carolina 
(not specified) 

4. Local agency in 
Guam (not 
specified) 


1 . Missing data; selection bias 

2. Lack of control group; 
missing data 

3. Lack of control group 

4. Selection bias 


Reifsnider and Eckhart, 
1997 


Duration of breastfeeding increased after prenatal 
nutrition education classes focusing on 
breastfeeding. 


WIC clinics in three 
rural Oklahoma 
counties (1986) 


Lack of control group 


Tuttle and Dewey, 1 995 


Breastfeeding initiation rates increased among 
Hmong WIC participants after a culturally sensitive 
prenatal breastfeeding class and prenatal and 
postpartum counseling. 


Seven WIC clinics in 
three California 
counties (1991-1992) 


^ Selection bias; 

* measurement error 
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Demonstration Studies: Major Findings, 
Scope, and Major Limitations 


(Continued From Previous Page) 


study 


Major finding(s) 


Scope (timeframe) 


Major llmitation(s) 


Shaw and Kaczorowski, 
1999 


Breastfeeding initiation and duration increased after 
counseling and support provided by WIC 
participants, trained and paid as peer counselors. 


WIC programs in nine 
West Tennessee 
health departments 
(1996-1997) 


Measurement error 


Grummer-Strawn and 
others, 1997 


Clinics with paid peer counselors had higher rates 
of breastfeeding initiation than clinics without peer 
counselors. Clinics with a lactation specialist or 
consultant, and peer counselors, had higher rates 
of breastfeeding initiation than clinics with only peer 
counselors. However, the benefits of lactation 
specialists were offset when peer counselors spent 
at least 45 minutes with individual participants. 


Fifty-one WIC clinics in 
Mississippi (1989- 
1993) 


Selection bias; 
missing data 


Referrals 


Birkhead and 
others, 

1998 


Children were 5.5 times more likely to be 
immunized, and immunized more rapidly, at WIC 
sites where staff escorted children to a pediatric 
clinic in the same facility for immunization. Children 
were almost 3 times more likely to be immunized, 
and immunized more rapidly, at sites with a 
voucher/check incentive. (Until immunization, a 
family must visit, the clinic monthly, rather than 
every other month, to pick up WIC voucher/checks.) 


Six WIC sites in New 
York City (1991) 


Lack of control group 


Hoekstra and others, 
1998 


Immunization rates increased at sites with a 
voucher/check Incentive (until children are 
immunized, a family must visit the clinic monthly, 
rather than every 3 months, to pick up WIC 
voucher/checks). 


Nineteen WIC sites in 
Chicago (1996-1997) 


Lack of control group; 

measurement 

error; inappropriate data 

analysis 

techniques 


Hutchins and others, 
1999 


Vaccinations increased at sites with vaccination 
screening and a voucher/check incentive (until 
immunization, a family must visit the clinic monthly, 
rather than every 3 months, to pick up WIC 
voucher/checks). 


Seven WIC sites in 
Chicago (1991-1993) 


Inappropriate data analysis 
techniques; selection bias . 
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